The present invention relates to digital television and, more particularly, to a system for displaying electronic documents synchronized to the audio/video content of a digital television program.
A digital television system includes a transmitter and receiver pair. The transmitter system includes subsystems to receive and compress digital source data; the data streams representing a program's audio, video, and ancillary data elements; multiplex the data from the several source streams into a single transport bit stream; and transmit the data to the receiver. At the receiver the transport bit stream is demultiplexed into its constituent elementary data streams. The elementary data streams are decompressed and the audio and video data streams are delivered as synchronized program elements to the receiver's presentation subsystem for display as a coordinated television program. The digital television system is described by the ATSC Digital Television Standard, Advanced Television Systems Committee, Doc A/53, 12 Apr. 1995, 16 Sep. 1995, incorporated by reference herein. The system employs the MPEG-2 systems and video stream syntax of ISO/IEC IS 13818-1 & 2, Information Technology—Generic Coding of Moving Pictures and Associated Audio Information, International Standards Organization, First Edition, 1996, 15 April 1996 and 15 May 1996, respectively, 16 Sep. 1995, incorporated by reference herein. Audio coding is in accordance with the Digital Audio Compression Standard (AC-3), Advanced Television Systems Committee, Doc A/52, 16 Nov. 1994, 20 Dec. 1995 16 Sep. 1995, incorporated by reference herein.
In addition to the basic audio and video program content, the digital television system provides for transmitting ancillary data services. Ancillary data services can include control data; conditional access control data associated with limited access to scrambled signals; and data associated with program audio and video services such as “closed captioning,” emergency broadcasts, and an optional program guide containing information about the programs being transmitted or to be transmitted. The suite of ATSC digital television standards will also include a specification for data broadcast services to be drafted by the T3S13 working group of the Advanced Television Systems Committee. The data broadcast services specification is expected to include mechanisms for synchronization of broadcast data services with the audio/video content of a television program.
Some ancillary data services for digital television may be facilitated by transmitting electronic documents similar to those transmitted over the World Wide Web and displayed by a web browser. These documents could be similar or identical to the HyperText Markup Language (HTML) formatted documents delivered over the Internet. Proposals for future revisions of HTML include a HyperText Markup Language for broadcast applications (bHTML) as described in “A Modular Hypertext Markup Language for Broadcast Application”; Draft 4; Wugofski, Ted; Over the Moon Productions; 1998. Among the changes proposed to make HTML more suitable for broadcast applications is the addition of structures related to time sensitive delivery of documents. Heretofore, HTML formatted documents may be transmitted using delivery time insensitive asynchronous protocols, such as the Internet protocol. For digital television it is desirable that documents are also capable of being displayed in a time sensitive manner. A document might be displayed in a synchronized mode, that is, the display of the document is coordinated with certain audio or video program content. On the other hand, the document might be displayed in a synchronous mode where the display of the document occurs at a specified time but the document's display not synchronized to the program content being displayed at that time.
To facilitate time sensitive delivery of HTML formatted documents temporal structures have been proposed as additions to the bHTML language. A paper; “Synchronization and Transition in Broadcast HTML” Addendum Version 1.0 to bHTML Draft 4; Wugofski, Ted; Over the Moon Productions; 1998; proposes expanding the bHTML syntax to include features related to timing, synchronization, and the transition of objects entering and exiting the display. Using the proposed timing features the author of a bHTML formatted document could specify in the document whether display of a HTML object would be based on time. To facilitate time based operation, the proposal includes tags which permit the document's author to identify a clock source and specify the reference value of that clock source when the time based event (for example, “start display”) is to occur.
The proposal identifies the system clock of the platform executing the decoding and rendering of the document as the default clock source. While the digital television receiver includes a clock, the present inventor has realized that it is not sufficiently accurate to be useful in close synchronization of the display of documents and program audio/video content. Further, potential differences in local time at the receiver, the transmitter, and the location where the program's content is authored would make synchronization of audio/video and data services based on the receiver's “clock” time very difficult.
The bHTML timing and synchronization proposal permits the document's author to identify a clock source other than the system clock. These clock sources could include a frame rate, a field rate, or a presentation time stamp. The digital television system relies on a 90 kHz MPEG-2 system clock for synchronization of the encoder and a decoder of the system. The digital television receiver reconstructs the system clock from Program Clock References (PCRs) which are transmitted with the data stream. In addition, presentation time stamps (PTS) are inserted in the elementary data streams before transmission to synchronize the display of audio and video elements of the program. The PTS is included at regular intervals in the header of video and audio data packets when the packets are encoded. The decompression and presentation systems of the receiver utilize the time stamps to synchronize the display of the reconstructed video and audio program elements. According to the bHTML proposal, a PTS reference value could be specified in the document. When the referenced PTS value is detected in the data stream, the document would be displayed.
While the bHTML proposal provides a structure internal to a document for specifying a PTS value, it does not suggest how synchronization might be implemented in a digital television system. The PTS is an encoding of absolute time, synchronous to the clock of the video or audio encoder, which is inserted into the elementary data streams of the program elements by the program multiplexer of the transmitter. The PTS values are only meaningful if the internal time base of the system encoder is known. In other words, the PTS does not specify time in any usual sense. The author of a document will not be able specify a PTS reference in the document when the document is authored because there is no way of determining the value of the PTS at some time in the future when the audio or video content to which the document is to be synchronized will pass through the program multiplexer. On the other hand, the program multiplexer does not have the capability or the time to parse the HTML document and insert a PTS reference in the document after the correct reference is determined.
The document could be transmitted asynchronously and stored at the receiver. When a special synchronization mark in the program video was detected, the document could be retrieved from storage and displayed. However, this would require parsing and buffering of the video to detect the synchronization mark and retrieve the document in sufficient time to be synchronized to the datum in the video. This would complicate the receiver and significantly increase its cost.
Since the PTS is available in the data stream at the receiver, a data presentation engine could recover the PTS for the data packet that contains the document and cause the document to be displayed at the appropriate time. However, this data presentation engine would be significantly different from the standard World Wide Web browser which can interpret the HTML tags within a document but is incapable of retrieving the PTS from the data stream for use in synchronizing display of the document.
What is desired, therefore, is a system for synchronizing the display of bHTML coded documents transmitted as part of the synchronized data services of a digital television system utilizing the proposed functions of Broadcast HTML.